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Implicit in the approaches being taken by current efforts to create intelligent 
computer-based instruction is the notion that curriculum is almost an epiphenomenon of 
knowledge-driven instruction. Early computer-based instruction had little conhol 
structure other than an absolutely rigid curriculum and was insensitive to the subtleties 
of different students' partial knowledge. As a result, there was a reaction in the 
direction of representing the students 1 knowledge as a subset of the target or goal 
knowledge to be taught and simply deciding de novo after each piece of instruction what 
piece of missing knowledge to teach the student. I am convinced that goal knowledge is 
as important to intelligent machine activity as it is to human activity, and that it also 
must be well understood and explicitly represented in an instructional system if that 
system is to be successful in fostering learning.! This report presents an architecture 
for representing curriculum or goal knowledge in intelligent tutors and is thus a first step 
toward a theory of curriculum that can inform the design of such systems. To illustrate 
one way in which such a theory can sharpen our ideas about learning and instruction, the 
later part of the report focuses on the concept of prerequisite that is the basis for 
existing computer-assisted instructior and shows how that concept has been inadequate 
in the past. A new approach, in which the prerequisite relationship is always dependent 
on the instructional subgoal (curriculum) context, is introduced. 

1.0 Current Practice 

Programs that preceded the entrance of artificial intelligence into instruction 
prespecify the content of lessons. In some cases, the order of the specific lessons to 
which a student is exposed is computed as instruction proceeds. However, the content of 
a lesson, in terms of knowledge it is trying to teach, tends to be fixed. In currently used 
programs, lesson assignment is viewed as a more-or-less knowledge-free subgoaling 
problem. We have a list of things to be taught, and we teach each in turn. If instruction 
is unsuccessful, we try again. The assignment of lessons can occur in several different 
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ways. The frame approach organizes instruction into very microscopic units, 
characteristically one screen "frame" each in size. Each frame contains some 
instruction, some means for testing the student's performance, and decision rules for 
deciding, on the basis of that performance, which frame the student should see next. 
The pretest-posttest approach is organized at a higher level, the level of tne lesson. 
Prior to each lesson, a pretest is given. If the pretest is passed, the lesson is skipped; 
otherwise it is presented. After each lesson a posttest is given. If the student passes the 
posttest, he goes on to take the pretest for the next lesson. 

There is a certain amount of inefficiency in the pretest-posttest approach. In many 
cases, the student spends most of his time taking tests, and many testing methods are 
not very effective instruction. More important, there usually is not much difference 
between the lesson taken initially by a student and the one he receives if he fails the 
posttest and is recycled. A number of efforts have been made in specific programs to 
assure that novel material is presented, or that the repeated lesson is taught more 
slowly, with more examples and more practice. However, in terms of content, it is the 
same old lesson, being repeated again. As we shah see, this may be a fundamentally 
incorrect approach to teaching. 

There are many rationales for the pretest-posttest approach. Of these, perhaps the 
strongest is the learning hierarchy theory of Robert Gagne (1962), Gagne gave very 
clear directions for deciding on the content and sequencing of instruction, and these 
directions continue to have strong influence on the design of instruction and training 
today. The basic approach is to start with the capability that is the goal of the training 
being designed.2 One is then to ask the question 

What kind of capability would an individual have to possess if he 
were able to perform this task successfully, were we to give him 
only instruction? (Gagne, 1962, p. 356) 



That is, what does the trainee have to know so that simple verbel advice is 
sufficient to get him to apply that prior knowledge to the task at hand? This 
methodology can, of course, be applied recursively to further decompose the target 
capability into smaller and smaller prerequisite capabilities. It is, after all, nothing but 
the generic problem solving method of progressive refinement, splitting a complex goal 
into several pieces, splitting the pieces into still smaller pieces, etc. This approach 
works as long as we pick as units pieces that are small and coherent enough to cover in 
single lessons and as long as there are no dangerous interactions between achieving one 
lesson subgoal and achieving another. 

These two criteria of subgoal internal coherence and linearity are extremely 
important to the success of the method of progressive refinement. By internal 
coherence, we mean that the subgoal can be achieved sensibly by itself, without 
duplicating effort across subgoals unnecessarily. For example, if we decided to mow the 
lawn by dividing it into a checkerboard of square regions, first mowing every other 
region, and then mowing the ones in between, our refinement of the lawn task into doing 
first the red squares of the board and then the black ones wastes effort. We must move 
over the entire lawn twice instead of once. Further, lifting the mower to avoid cutting 
certain sections which are being postponed for later is also unnecessary work. 

Note that internal coherence is a function of the specifics of subgoal contents and 
not a general principle alone. For example, while the checkerboard approach to mowing 
the lawn is inefficient, making two passes over the lawn, one to mow and one to remove 
weeds, may be quite sensible. This is because the micro-acts of pulling one weed and 
mowing a small patch are incompatible, producing extra thinking and physical work if 
they must be continually alternated, while the micro-acts of mowing successive small 
regions are quite compatible, saving work over doing one patch, then mowing it, doing 
the next, etc. It might be argued that the current approach of teaching subtraction of 
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f r two-digit numbers and then waiting as long as a year before taking up subtraction of 

f . three-digit numbers also violates the internal coherence rule* 

f The linearity criterion is also important. For example, if we want to build a house, 

we can split the task into laying a foundation, putting up walls, and putting on a roof. 

■ Only one order of those three steps make sense. The other orders fail. Some orderings 

are impossible: one cannot put on a roof if there are no walls to support it yet. Other 

\ orderings involve early steps that interfere with later ones, e.g., putting up the walls 

! before the foundation. Linearity problems can also arise in learning. For example, if 

one requires students to type their essays beginning in fifth grade, and typing is not 
taught until seventh grade, then there will be some conflict. Students will acquire, on 
their own, patterns of typing that conflict with the behaviors used in efficient typing 
methods; there could be negative transfer. 

Gagn6 (1971) undertook to prescribe some principles of learning that would help 
instructional designers achieve internal coherence and linearity in their development of 
hierarchical goal structures (which he called learning hierarchies) for training courses. 
He developed a variety of specific learning forms that could be used to constrain the 
parceling of pieces of instruction into separate lessons or curriculum subgoals. Implicit 
in his work is the principle that, in learning, the whole is more than the sum of its parts. 
That is, the lowest-level subgoals in a goal hierarchy do not, as a group, contain all the 
knowledge implied by the highest-level goal. The instructions that are given as subgoals 
are assembled into larger units of capability that result in new learning. This new 
learning is not part of any subgoaTs knowledge. Rather, it is emergent when multiple 
subgoals are combined, just as a theorem in geometry is not present \n the premises from 
which it is derived but is, rather, new knowledge. 

Unfortunately, the belief that the whole is simply the sum of its parts has too often 
guided the task analyses that have generated training curricula. Rational task analysts 
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have tended to look at an instructional subgoal, intuitively decide whether it can be 
taught in a single lesson, and, if not, split it into a few subsubgoals each to be treated by 
a separate lesson. Gagne's criteria, that we should be able to provide the "glue" needed 
to tie together the pieces of knowledge from prerequisite lessons using only simple 
instructions, and that these instructions should be based upon a theory of learning, are 
generally not given adequate consideration. For example, designers of training for 
technicians make intuitive decisions that a good technician must know some of the 
theory of operation for the devices he maintains and that he must know some specific 
rules for fixing those devices. A current technical curriculum therefore may cover 
primarily decontextualized theory and device-specific operating algorithms. No effort 
appears to have gone into determining the cognitive glue that allows concepts learned in 
one context to be applied in another, such as finding a fault in a nonworking device, 
which may require problem solving heuristics rather than specific algorithms. 

Intelligent computer-assisted instruction attempts to represent all the knowledge 
that constitutes the expertise that is to be taught. Interestingly, though, it generally 
does not possess an explicit curriculum based upon a theory of learning and instruction, 
either. Where conventional instruction has an explicit curriculum but fails to have an 
explicit and complete representation of the knowledge that is to be taught, intelligent 
instructional systems have tended to represent the target knowledge explicitly but not 
to represent explicitly that body of knowledge that specifies the goal structure for 
instruction, the curriculum. For example, the WEST tutor (Burton & Brown, 1982) 
contains a method for determining how close to optimal a player's performance is and a 
set of issues to be considered. These issues constitute part of a curriculum knowledge 
structure but fail to have any relational structure tying them to each other or to a 
representation of target knowledge. In other ?ases, such as the geometry and lisp tutors 
being developed by John Anderson (Anderson, Boyle, Farrell, <5c Reiser, 1984), there are 



problem sequences which are preset, but again there is no explicit representation of 
curriculum knowledge. 

To summarize., traditional computer-based instruction, whether organized into 
frames or into larger lesson units, tends to have an explicit representation of curricular 
structure, though often a shallow one, and, at best, only an implicit representation of the 
knowledge being taught, while intelligent instructional systems developed to date have 
explicit representation of the target knowledge but at best only implicit representations 
of the curriculum knowledge, the scope and sequencing of lessons. 

2.0 Weaknesses in Current Approach 

We can now restate and elaborate two problems inherent in current approaches to 
instructional design. First, there is no clear method for differentiating how to present 
material to remediate a problem discovered after a lesson has been taught from how it 
should be presented when taught initially. Second, the knowledge that represents the 
"glue" connecting the contents of related lessons is not clearly specified, nor is it 
assigned to be part of the content of any specific lesson. I consider each of these 
problems in turn. 

2.1 Redo is the Only Strategy It Supports 

In current training systems, the curriculum is at least implicitly a goal structure. 
One proceeds to teach the prerequisites (or subgoals) for a given lesson before teaching 
that lesson itself. The student should always be able to infer the missing knowledge that 
integrates those pieces of prerequisite knowledge into a broader skill. However, this 
doesn't always work. Lesgold's Two Fundamental Laws of Instruction (shown in Table 1) 
often apply, resulting in incorrect decisions about whether or not a trainee has mastered 
a given lesson. A lesson can appear to be mastered although the knowledge that has been 
acquired is too specific and cannot transfer from the context of the prerequisite training 
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to the context in which it must be used (missing "glue"), resulting in the circumstance 
described by the First Law, On the other hand, a lesson can appear to be unlearned 
because the context of testing does not match the context of instruction, even if the 
learned knowledge is adequate to the contexts for which the lesson is prerequisite. The 
result is the circumstance described by the Second Law, For example, there are 
mechanics whose formal knowledge of electrical principles is not sufficiently developed 
to pass tests but who know more about what goes on in a car's electrical system than 
many people who have studied physics. 



Table 1 

Lesgold's Two Fundamental Laws of Instruction 
First Law: Not everyone who passes a test on a topic knows what appears to have been tested 

Second Law: Not everyone who fails a test on a topic lacks the knowledge that appears to have been 

tested. 



Because we are never able to establish with certainty that a lerson has been learned, 
and because excessively high criteria for posttesting can be very wasteful of 
instructional time (since the student could fail but still have adequate knowledge), it is 
inevitable that occasions will arise in which it was assumed that a student has mastered 
a prerequisite when in fact he has not. It is important to note that this will occur 
whether the criteria for passing out of a lesson are prespecified or determined through 
some inferential process as th\ lesson is being taught, whether they are superficial (a 
cutoff score on ft test) or deep (based upon a detailed student model fitted to all of the 
student's performance in the relevant recent past). 
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When *^.e student does have to be given remediation because the assumptions of 
prerequisite knowledge have proven wrong, current systems generally replay the same 
instruction that did not work the first time. Sometimes different problems are assigned 
as examples or for practice, but they are generally of the same type as were used before. 
Sometimes the lesson proceeds more slowly, taking smaller steps and providing more 
practice at each step. In a few of the most recent intelligent computer assisted 
instructional systems (e.g., Burton ic Brown, 1982; Bunar, 1985), it is possible to estimate 
which specific pieces of knowledge targeted by the lesson ure most likely not to hav^ 
been learned and to concentrate on those. However, in every case, the goals of the 
lesson can and should be adapted to specific needs that arise only during remediation. 

It should be noted that a number of theorists have proposed the view that learning in 
a domain is a process of successively replacing primitive conceptions, or personal 
theories, of the domain with more advanced constructions of it. This view subsumes the 
important insight that the knowledge structure of the student, rather than simply being 
incomplete, may actually be wrong, that he may hold a misconception, a different and 
conflicting theory of the task domain from the one the trainer wou, .ike him to have 
(for example, see Carey, in press; Glaser, 1S>,4.; Shaughnessy, 1977; Young <5c O f Shea, 
1981). The approach I am taking in this report is complementary to this view, 
concentrating on a somewhat more microscopic level of analysis. 

2.2 No One Is Responsible for the Likeliest Failures of Instruction 

A second problem with current approaches to remediation is that they 
fundamentally ignore the nonlinearities between lesjons, the "glue" that holds lessons 
together. This is a problem particularly when responsibility for different parts of a 
training regimen is divided among multiple instructors. There if always a tendency for 
the content of a lesson to be abstracted to its co-e, both in teaching it and in deciding 
how to test it. Consequently, the amount of between-lesson "glue 11 for which none of the 
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course instructors takes responsibility can be substantial. Even worse, such division of 
responsibility is most common in technical training courses, where the trainees are often 
less likely to be facile at inferring the missing content. However, it occurs between 
grades in school and between courses in college curricula as well. 

The loss of the fringe content between lessons, the "glue," produces a variety of 
pathologies that we see every day. The teacher in third grade feels tha* the only reason 
students do poorly in her class is that the second grade teacher failed to teach what was 
required. The second grade teacher points to high test scores and disclaims 
responsibility. The trainer providing on-the-job practice to technicians claims that they 
do poorly only because they were not taught fundamental principles of electricity, while 
the instructor for the course on fundamentals of electricity has lots of test data to show 
that they learned everything in the curriculum. Of course, what they were likely to 
forget after testing is exactly the fringes of the knowledge they were taught, the 
relations that w glue" it to the content of other lessons. 

Overall, both of the above-cited problems seem to arise because there is a lack of 
distinction between the content of training and the curriculum or goal structure for 
training. Merely checking off subgoals as they are taught fails to take account of the 
tendency of declarative knowledge to suffer high forgetting at its fringes, to shrink to a 
coherent, highly-interconnected core. Simply reteaching prerequisite lessons when 
problems arise later fails to take account of what has already been learned and what has 
just been shown specifically to be weak. 

It. the next section, I introduce a knowledge architecture for intelligent tutoring 
systems that has driven considerable current work on intelligent instructional systems 
that Jeffrey Bonar, Robert Glaser, and I (cf. Bonar, 1935; Glaser, Lesgold, <5c Lajoie, in 
press) have been conducting. The architecture's components are still being shaped by 




13 



various projects each of us is conducting, but the basic ideas are well enough evolved to 
help supply what is missing from current instructional systems. 

3.0 The Structure of Knowledge in an Intelligent Tutor. 

As discussed above, a fundamental problem with existing architectures for 
instructional and training systems is that they fail to explicitly represent either the 
knowledge that they are designed to teach or the curriculum (goal structure) for teaching 
it. In setting out to develop an architecture that represents both content and 
curriculum, one quickly discovers that there are other issues that must be considered. 
First, there is other knowledge that a good teacher looks for in the student, knowledge 
we might call aptitude or metacognitive skill. That is, some students are more able to 
learn with facility from particular forms of instruction than others, and awareness of a 
student's learning capabilities can well shape the instructional approach a good teacher 
takes. Second, there are more domain-specific capabilities that represent both the broad 
outcomes desired from the training and some specific capabilities for learning. For 
example, in a course on troubleshooting, general knowledge of electrical principles is 
both a useful prerequisite and something that should be enhanced by practice in finding 
faults in circuits. 

These needs suggest that the knowledge in an intelligent tutor must be of three 
different types: (a) curriculum knowledge, a subgoal lattice of lessons connected by the 
prerequisite relation; (b) a representation of the knowledge to be taught, from which 
explanations and student models can be generated; and (c) a representation of the more 
enduring characteristics (metaissues) to which the instruction should be sensitive. Figure 
1 shows the architecture symbolically. 
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Figure 1 

The Three Layers of an intelligent Tutor 



3.1 The Knowledge Layer 

The knowledge layer, highlighted in Figure 2, should contain a representation of the 
knowledge the system is trying to teach. One way to think about that knowledge is that 
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it is a model of expert capability in the domain. Such knowledge includes both 
procedures and concepts (i.e., both procedural and declarative knowledge). Some 
constraints on its structure can be inferred from several things we know about human 
expertise. For example, we know that experts, in contrast to novices, tend to represent 
problems according to the underlying situations they involve, according tc their deep 
structure, whereas novices tend to have more superficial representations (Chi, Feltovich, 
& Glaser, 1981; Larkin, McDermott, Simon, & Simon, 1980). We also know that, in 
contrast to intermediate-level performers, experts tend to know exactly what to do in a 
given situation rather than being dependent on inference from first principle? (Chase ic 
Simon, 1973; de Groot, 1965). That is, they are able to represent the situation more 
completely and richly and then to invoke the precisely appropriate method for dealing 
with it. In order to do this, experts' knowledge must be richly interconnected and, to 
some extent, redundant. 

Before discussing how to deal with this problem, let me pause and give an example 
or two to illustrate the importance of the bridging connections between coherent bodies 
of knowledge. Consider the field of medicine. It is driven by several sciences: 
physiology, biochemistry, pharmacology, and even physics and chemistry. The relative 
roles of different portions of its scientific backing will differ for different disease 
problems. For this reason, medicine is organized into specialties, each of which is 
internally very coherent. The ties between these specialties are much more complex 
and, relatively speaking, ad hoc. 
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Figure 2 
The Knowledge Layer 



A friend who is an attorney is handling a malpractice case involving a man who 
arrived at a hospital with severe back pain and several other symptoms. A very 
inexperienced intern was in the emergency room. He sent ihe patient to the orthopedics 
department. The specialists there found no skeletal problems and, after a lengthy 
examination, sent him back to the emergency room again. It then occurred to the intern 
that the man might have a tumor producing the pain. So, he was sent to oncology. While 
waiting for his turn there, he collapsed and died of a renal artery aneurysm. In essence, 
the diagnosis process, decentralized according to the primary joints in the body of 
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medical knowledge, failed to make adequate use of the linking knowledge that might 
have led from the symptoms, which seemed to involve the back or the organs near the 
painful spot, to knowledge of various sorts of vascular problems. What was missing from 
the process was exactly the ad hoc interconnections between the conventional units of 
knowledge. Part of the specialty of emergency medicine is knowledge of the limits of 
the diagnostic processes which different specialties use, i.e., knowledge of what happens 
at the "fringes" between specialized diagnostic approaches. 

One can find other less-dramatic examples throughout the curriculum. For example, 
in teaching elementary arithmetic, we teach children about place value, concentrating 
on the ones, tens, and hundreds places in numbers. We also teach them algorithms for 
addition and subtraction of multicolumn numbers. Not all of the ties between these two 
related pieces of knowledge are explicitly taught, and not all seem to be universally 
learned (as demonstrated by the BUGGY line of research; Brown <5c Burton, 1978; Brown 
& Van Lehn, 1980; Van Lehn, 1983). Some of what is not universally learned also has the 
character of being in the gap between the two pieces of instruction. 

How do we deal with such gaps? One approach is a small extension of the original 
Gagne learning hierarchy ideas. The lowest levels of lessons in such a hierarchy 
correspond to the regions of the knowledge layer into which the total body of expert 
knowledge has been split. Higher levels of lessons are more than just the sum of what 
was taught in the lower-level lessons. They have the specific task of assuring that the 
conceptual glue between the lower-level pieces is acquired. It is in this sense that 
teaching the whole of a body of material is more than just teaching its parts; the goal for 
the whole includes not only the parts but also a specific focus on the ties between those 
parts. 
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Figure 3 

Goal Hierarchy for Basic Resistor Network Laws 
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3.2 The Curriculum Goal Lattice Layer 

The curriculum goal lattice layer is the central layer of the proposed architecture. 
As the goal structure for the instructional system, it is, more or less, in control of the 
system. Ordinarily, goal structures are trees, representing the progressive 
decomposition of each layer of subgoals into still smaller subsubgoals. An example tree 
is shown in Figure 3. It shows the decomposition of a basic course in resistor network 
concepts into two main goals, knowledge of Ohm's Law and Knowledge of Kirchhoff's 
Law. Those goals are then broken into subgoals, which are then decomposed further. 
The diagram becomes rather complex visually, but its underlying structure is still 
straightforward: each subgoal is either a lesson that can be taught completely as a unit 
or it is further decomposable into subsubgoals. 

This kind of goal structure is exactly the sort of concept that Gagne was introducing 
in his discussions (see above) of learning hierarchies. Further, he felt that psychological 
laws of learning would determine when subgoals had to be further subdivided and when 
they could be taught as single lessons. His analyses were in terms of the verbal 
association theory prominent at the time. More recently, Van Lehn has advanced at 
least one different sort of criterion for deciding on such a subdivision, namely that a 
single lesson should not require the student to learn a rule with disjunctive conditions 
(Van Lehn, 1983). Clearly, if the structure of curriculum is simply a subgoal tree, we are 
well on the way to understanding how to develop such a tree and how detailed its 
xborization must be. 

However, in the first efforts, by Jeffrey Bonar, his students Cynthia Cosic and 
Leslie Wheeler, and I, to employ a curriculum goal hierarchy in an intelligent tutor, 
things were not as simple as we had hoped. For example, while one valid way to think of 
the resistor networks course is in terms of scientific laws presented, which leads to the 
decomposition shown in Figure 3, there are other equally valid ways. For example, one 



might start with the basic measurable properties of such networks: current, 
electromotive force (voltage), and resistance. This leads to a goal lattice such as that 
shown in Figure 4. What is noteworthy is that the lowest level units in the tree, the 
simple lessons, are the same as in Figure 3, but the organization into higher-order goals 
is entirely different, and the apparent purpose of the course may be different. 
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Figure 4 

Goal Hierarchy for Basic Resistor Network Measures 
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So fa.% in our own work, we have found four different viewpoints on the instruction 
that we want to present, sa^h of which gives rise to a hierarchy that projects onto the 
same simple lessons. Figure 5 shows chesc viewpoints. We can partition our lessons into 
those that deal with series circuits and thcc* that deal with parallel circuits, quite 
reasonable given that students often have different conceptual problems with parallel 
circuits. Or, we can partition our lessons ac tiding to the type of problem we present to 
the student: qualitative problems, quantitative problems, and problems that involve 
making a relative judgment about the one circuit relative to another. There are, cf 
course, also the two viewpoints discussed above, laws and measurable properties. 
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Viewpoint* on Resistor Network instruction 



This leads us to a new view of the structure of curriculum knowledge in the 
knowledge base of an intelligent tutor. The curriculum knowledge has the structure of a 
goal lattice. There are a number of viewpoints on the goals of the instruction. With 
respect to each viewpoint, one can identify a subset of the curriculum lattice that is a 
true subgoal tree structure. So, from any specific point of view, there are clear 
pathways that determine the sequencing of instruction, though of course there are 
alternate approaches to such sequencing. For example, one can proceed depth-first. In 
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the case of Figure 4's viewpoint, this would mean perhaps doing all the lessons relating to 
current, then all those relating to voltage, and finally all those relating to resistance. 
Or, one might proceed breadth-first, going through all the lowest -level lessons, then the 
next level, and so on. There may be individual differences in aptitude or preference for 
these two approaches. 

Of course, when all of the viewpoints are considered at once, there is much more 
complexity to the task of deciding what the appropriate sequencing for the lessons of the 
curriculum should be. To some extent, the decision can be made on the basis of rational 
task analysis, but our experience has been that empirical work driven by cognitive theory 
is often necessary (Lesgold et aL, 1986). Some of the lessons will tend to be difficult and 
others easy. By taking the various viewpoints, it should be possible to organize 
knowledge about lesson difficulty sufficiently to use it in deciding on appropriate 
orcJcrings through the curriculum. Another approach may be to tell the student which 
lessons he is w eligible ff to take next, based on prerequisites completed, and let him decide 
for himself. As we shall see in section 4 of this report, there are even more 
sophisticated possibilities to consider in deciding how to handle sequencing. 

To summarize, the goal lattice layer is a lattice structure in which are embedded a 
number of goal hierarchies, each corresponding to a fundamental viewpoint on the task 
of teaching the course content. Figure 6 shows the goal lattice for the resistor networks 
course we are implementing. This multiple viewpoints approach, incidentally, has 
implications for what constitutes an appropriate course, in terms of the completeness, 
coherence, and consistency of its curriculum lattice. Presumably, the resistor networks 
course as shown in Figure 6 is a reasonably sensible selection of content for a course. 
The course is coherent, in that each simple lesson is relevant to all of the viewpoints we 
have taken, it is locally complete, in that each viewpoint seems to be completely 
teachable with the set of simple lessons we currently have implemented. It is globally 



complete to the extent that the viewpoints represented include all of the viewpoints 
routinely held by experts and any others that are important to learning the domain 
content. Finally, it is relatively consistent, in that the prerequisite relationships all run 
in the same direction. There are no cases where Lesson X is prerequisite to Lesson Y 
from one point of view while Lesson Y is prerequisite to Lesson X from another. 
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It is not inevitable that courses will have these three properties. For example, 
consider the sort of introductory psychology course suggested by most current textbooks. 
There is little sense of completeness; lessons are included simple because of marketing 
needs and current fashion. There is no sense in which the simple lessons at the bottom of 
the hierarchy represent the simple foundations of concepts. Similarly, there is little 
coherence; the differing viewpoints, social, behavioral, cognitive, clinical, do not cover 
the same set of underlying basic concepts. Finally, there may not even be consistency. 
From one viewpoint, it may be best to teach sensory physiology before teaching about 
mental imagery; from another point of view a reverse ordering may seem obvious. 

it seems appropriate to advance, as a hypotb^is for future research, that 
knowledge-driven instructional systems will work best and be most implementable for 
those courses which have more or less coherent, consistent, and complete goal 
structures. 

3.3 The Metaissue Layer 

The third layer of the proposed architecture for instructional knowledge is the 
metaissue layer. Once again, it is useful to recount some of our reasoning in deciding on 
the need for such a layer and on what should be in it. Initially, we were notivated by a 
single issue: the conflict between the sorts of data used by very good teachers to decide 
on how to proceed with a given student and the data and reasoning used by the few 
expert instructional systems that have been built. In giving students assignments, 
teachers tend to rely on very broad representation of aptitude combined with a detailed 
knowledge cf the curriculum. A child is "a good student," "a fast learner, 11 "good in 
math," and/or "on page 93 in the book." In contrast, intelligent instructional systems, as 
envisioned by artificial intelligence researchers (e.g., Burton & Brown, 1982; Goldstein & 
Carr, 1977) construct a detailed student model which represents the best guess about 
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exactly what a student does and does not know of the specific material targeted by the 
course, e.g., "borrowing across zero, 11 "adding single-digit integers," etc. 

There is at least modest evidence (Burton ic Brown, 1982) that teachers cannot 
determine microscopic representations of student knowledge status nearly as well as 
intelligent computer systems can, so the first hope for an approach that would be 
sensitive to the detailed specifics of changing student knowledge is probably an 
intelligent instructional computer system. This prompted us to think about how to 
represent aptitude data. We were, in this thinking, heavily influenced by the 
object-oriented approach we were taking (this approach will be discussed in Section 3.4 
below). Basically, we were led to the following point of view. 
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Attending to a specific aptitude or some other metaissue in shaping the activities 
presented for the trainee in any lesson is simply a special case of shaping a lesson 
according to a specific viewpoint. That is, just as we might attend to differences 
between series and parallel circuits in our resistor network tutor and expect some 
students to have trouble with parallel circuits even after they have mastered series 
circuits, we could attend to differences among students in, say, reading ability or verbal 
facility of the student* and thus tailor our teaching to each student's capabilities seen 
from the verbal-facility point of view. This has led us to the architecture shown in 
Figure 7, in which the metaissue layer is simply the collection of goal nodes that are the 
origins of various viewpoint hierarchies embedded within the curriculum lattice. 

3.4 Lesson Objects 

So far, we have rather mysteriously presented descriptions of various structures of 
knowledge, implying that it is organized into lessons that can be considered from a 
variety of different viewpoints. Also, we have suggested that, somehow, the lessons or 
subgoals of the curriculum are connected with a representation of the knowledge they 
are trying to teach. Nowhere have we said just what a lesson is, just what the structure 
within one of these graphs might be like. To this I now turn. 

Our fundamental approach to designing architectures for intelligent instructional 
systems is object oriented. That is, we see the design task as one of specifying a set of 
intelligent fragments of computer program and then orchestrating the interactions 
among these fragments. This approach originated with Smalltalk (Goldberg <5c Robson, 
1983), a language developed in the course of trying to determine ways in which powerful 
personal workstations could change education. In conventional computer programs, the 
primary means of controlling the order in which computations take place, the task 
discipline, is by the sequencing of instructions. In object-oriented programming, control 
is passed when objects, "entities that combine the properties of procedures and data 



since they perform computations and save local state" (Stefik & Bobrow, 1986), send 
messages to other objects* 

So, for example, the way in which our resistor network tutor might be started is for 
the student to point to a box on the screen that says Start. That box would actually be a 
menu operated by an object. The object would perhaps respond to the student's action by 
telling one of the metaissue nodes in the tutor to teach the student. For example, if the 
approach favored by the designer were to teach primarily the relevant laws of electrical 
circuits, the menu object might send the Laws object (see Figure 3) a message to teach 
everything for which it is responsible. The Laws object would, in turn, ask its first 
prerequisite object, Kirch hoffsLaw, to act, and that object might in turn tell the I + Parallel 
object, which teaches that current sums across the branches of a parallel circuit, to act. 
The I + Parallel object would then send messages in turn to its two prerequisites, dealing 
with current summing over branches and with the notion that the current in a branch of a 
parallel network is always less than the current passing through the network as a whole, 
to teach their stuff. At each level, when one subgoal of a goal was satisfied, the next 
would be sent a message to act, and so on. 

This requires that each object contain all the data and all of the methods needed to 
completely achieve the goal to which it corresponds. This is not as cumbersome as it 
may sound; it is not necessary for each object to be a complete instructional computer 
program. Rather, objects can "inherit" some of their methods from higher-level objects. 
So, for example, if there are many objects that should teach their content via an 
exploratory electrical circuits simulation environment, they can all have a pointer to a 
single higher-level object that includes the program for such a simulation. Each object 
using the simulation might specialize it aither by setting the values c* variables to which 
the simulation program refers or by including specializing information in a message it 
sends to the simulator when it invokes that approach. The object-oriented approach is 
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valuable largely because it provides for a clear, understandable, and flexible means of 
assuring that the goals an instructional program has to achieve are clearly delineated and 
clearly "tasked" by relevant pieces of program code, even if the content to be taught 
under various circumstances must be determined dynamically. 

Table 2 lists the contents that each goal lattice object must have in the kinds of 
instructional systems we are currently building. The list is split into two parts. 
Declarative knowledge is the data that an object must have (or be prepared to 
accumulate during interactions with the student), what it is able to know. Procedural 
knowledge is the set of methods or programs that the object must have, what it is able to 
do. Each entry represents a specific kind of knowledge that must be present either 
explicitly, by being included in the object, or implicitly, via a pointer to the knowledge 
as part of a "parent" object. 

Table 2 

The Contents of a Lesson Object 

Declarative Knowledge 

Variables that identify how a given lessons' goals relaxe to the goals of other lessons (i.e., which 
lessons are prerequisite to the current one). 

Variables that identify how the knowledge a lesson is trying to teach relates to the knowledge 
other lessons are trying to teach (pointers to the knowledge layer). 

Variables that represent the student's mastery of the knowledge the lesson intends to teach (the 
student model). 

Procedural Knowledge 

Functions (methods) that generate instructional interventions based upon the student model 
held by the given object, including both manipulations of the interactive learning environment 
(perhaps a simulated laboratory or a problem generator) and various forms of coaching or advising. 

Functions that decide if the given object is to blame for problems that arise while other lesson 
objects for which it is prerequisite are in control. 
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The declarative knowledge must include knowledge that places the object in the 
curriculum lattice, showing what its prerequisites are and also which objects assume it as 
a prerequisite. It must also include a specification of the specific parts of the knowledge 
layer (the representation of the target knowledge to be taught by the course) it is 
responsible for teaching. From this representation, it can provide explanations to the 
student. Finally, there must be a student model, a representation of which pieces of the 
target knowledge for the lesson the student appears to know and of the certainty of that 
diagnostic information. 

The procedural knowledge each object must have is of two primary types. First, an 
object must (again, either explicitly or implicitly) be able to teach its target knowledge. 
Second, it must be able to decide whether a student's failure to perform adequately in 
learning a lesson for which it is prerequisite might bi due to inadequate learning of its 
target knowledge. We call this blame-taking capability. The idea is that if things are 
going poorly in a lesson, the object teaching that lesson might ask each of its 
prerequisite objects to find out whether what it was supposed to teach is what the 
student is missing. With the curriculum structures discussed so far, that would seem to 
imply something like readministering pieces of the lesson posttest. However, there are 
more interesting possibilities to consider. 

The goal object lattice structure, as discussed so far, bears striking resemblance to 
current practice. If one looks at a current elementary or high school textbook, one finds 
that each topic, and the exercises associated with it, is treated only once. One problem 
with curricula that actually follow such books (and most teachers do) is that the 
conditions of applicability for pieces of fact and process that students are taught are 
never reliably delineated. Students too often form rules for carrying out problem solving 
that are perfect for passing a unit test but maladaptive in the long run. For example, a 
student in elementary school given a set of arithmetic word problems might learn that it 

30 



26 



isn't really necessary to read or understand the problem. If one finds words like 
altogether, one adds the numbers stated in the problem; if one finds words like less, one 
subtracts the smaller number from the larger; etc* Scanlon and O'Shea (in press) found 
similarly superficial strategies for use of specific equations that had recently been 
taught. 

The way to avoid, or at least eventually remediate, such superficial learning is to 
combine different types of problems, to assure that problems occur in a variety of 
superficial contexts, so that the successful cues for various actions are cues based upon 
deep understanding of the problem. Jeffrey Bonar has been working on an extension of 
the architecture I describe here that tries to do this. In essence, higher-level curriculum 
objects keep a list of lower level objects that have recently been taught. Occasionally, a 
few problems are created that require various unpredictable combinations of this 
knowledge for their solution. This forces the student to look more deeply at the problem 
situation and work, like an expert, from deep understanding rather than from surface 
appearances. 

However, the blame-taking problem becomes somewhat different in such cases. 
Rather than giving individual lesson posttests, testing for prerequisite knowledge in the 
original limited context used to present it, one wants to determine which pieces of 
knowledge that appear to have been mastered do not generalize to new situations. This 
appears to require a strategy of responding to a student's failure to handle a complex, 
multi-lesson problem either by giving hints or by giving a simpler problem, so that some 
of the candidate knowledge generalization failures can be ruled out. Work on how to do 
this is still proceeding; I mention it only because it may help give a sense of the 
character of blame-taking processes that we envision. 
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4.0 Context-Specific Prerequisite Content 

I turn now to the final issue I wish to address, the complex nature of the prerequisite 
relationship* In particular, I want to show that the domain knowledge for which a lesson 
object is responsible is specific to the curricular context in which the lesson is invoked. 
That is, the knowledge that should be presented by a lesson depends upon the context in 
which that lesson is taught. To do this, I must compare the core content of a lesson 
being taught for the first time with its remedial content. 

4.1 The Core Content of a Lesson 

The target knowledge of a lesson object can be thought of as a set of pointers from 
that object to nodes in the knowledge layer. The subset of the domain (or expert) 
knowledge layer defined by those nodes and their relations to other nodes will generally 
not have a sharp boundary, because expert knowledge is highly interconnected. As can 
be seen in Figure 8, some of the nodes in a lesson's target knowledge will be connected to 
each other, while others will be outlying orphans, whose operational meaning, that is, the 
set of connections from a, concept to node to other concept nodes, is defined primarily 
outside of the target subset. I use the term core content to refer to the subset of a 
lesson's target knowledge that is coherent, in the sense that its nodes are interconnected, 
with relatively few connections from a node to others outside that subset. 

When a lesson is taught initially, its cors content should be presented. That is, a 
coherent subset of the knowledge subsumed under the lesson should be taught. The 
density of detail in that coherent subset can vary with the aptitudes of the learner. 
Some learners should be taught all of the core content explicitly, while others can be 
expected to make at least the most direct and obvious inferences. In either case, it is 
impossible to teach explicitly the fringes of the target knowledge without also 
introducing knowledge outside the target subset, so it can be assumed that these fringe 
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pieces of knowledge have not been taught during the lesson's initial presentation. If a 
course, whether presented by human or by machine, is well taught, then the fringe 
knowledge for one lesson will be covered in another overlapping lesson or will be so 
immediately inferable from what is taught that it is optimal to assume that the student 
will learn it. Part of the artistry of curriculum design is to split the knowledge to be 
taught into pieces that cover the total set of target knowledge with no more overlap 
than the usual student will require. 
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Figure 8 

A Curriculum Object Subsumes a Region of the Knowledge Layer Including More than Core Content 



Further, there must be some mechanism for verifying how well the student is doing 
at learning what he is taught explicitly and completely and at inferring what it was 
hoped he would infer. In the design rr»y colleagues and I have been evolving, blame taking 
provides this mechanism. When a lesson is proceeding poorly, an effort is made to 
determine which prerequisite material was not well presented, and that material is then 
retaught. Further, arbitrary objects can be created dynamically from time to time 
whose task it is to compose problems that require synthesis of several different pieces of 
already-taught knowledge and to test the student with them. When such problems fail 
one or more of the lessons they were based on should be reconsidered (i.e., the 
instructor/machine should consider reteaching them). 

4.2 The Content of a Remedial Lesson 

When a lesson is retaught remedially, there will generally be information to guide 
the selection of content that should be emphasized. In contrast to the emphasis placed 
on core content when a lesson is originally taught, it is crucial to teach the knowledge 
that links the core content of the to-be-remediated lesson with the core content of the 
lesson whose failure produced the need for remediation. We can make this point clearer 
by resorting to a graphical representation. 

Look at Figure 9, which represents the interface between the goal lattice layer and the 
domain knowledge layer. Its point is that prerequisites are usually only partially 
overlapped by th** ? r superordinate lessons' domain knowledge. For example, the 
projection in the knowledge layer of Lesson A (Region a„ is only partly contained in the 
projection of Lesson C (Region c), for which it is prerequisite. It has a different overlap 
with the projection of Lesson B (Region b), for which it is also prerequisite. 
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Higher-Level Objects* Projections Do Not Necessarily Cover the Projections of their Children 



We can now proceed to define what content of a lesson should be taught when it is 
remediated. Basically, the emphasis should be on the nodes in the overlap between the 
projection of the superordinate lesson that failed and the projection of the prerequisite 
lesson that has taken blame. Figure 10 illustates the area of Goal A f s content that 
should be taught when A has taken blame for the failure of superordinate Goal B. 
Further, this overlap region should not be trimmed completely to produce a coherent 
core. Rather, its connections into the prerequisite lesson should also be pursued during 
remediation, and perhaps also its connections into the superordinate lesson that failed. 
This gives us a clear distinction between the lesson as originally taught, which 
emphasized core content, and the lesson as remediated, which involves contextually 
relevant context. 

I conclude by noting that this specification of what should be taught in remedial 
instruction is probably not foreign to the master teacher, who undoubtedly makes such 
determinations intuitively. However, instructional machines must have principles to 
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guide their performance, and the principles just stated seem reasonable candidates for 
inclusion in the teaching knowledge of such machines. In implementing this approach, I 
am sure that other candidates will also emerge. Finally, I suspect that the concerns we 
have had will be worth bringing to the attention of new teachers, who may not have a 
good idea of what the differences are between good remediation and simple repetition of 
instruction that failed the first time. 

I also note that I have not considered another principle that seems worthy, namely 
that instruction should build from strength. This is not because I disagree with that 
principle but rather because I have nothing new ready to say yet. A good instructional 
system, especially when remediating, will want to order the knowledge that is presented 
so that it builds from knowledge the student is known to have already. I hope that my 
colleagues and I will have something to say soon about how this should be done. 
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Footnotes 

*I thank David Merrill for making this clear to me in his comments after a 
presentation I made at an AERA meeting in 1983. 

2l mean to address both school instruction and technical training needs with the 
ideas presented in this essay. For ease of exposition, I shall use the term training to 
refer to both of these activities. 
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Carnegie-Mellon University 
Pittsburgh, PA 15213 

Library, 

Naval War College 
Newport, RI 02940 
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Center for tne Neurobiology 

of Learning and Memory 
University of California, Irvine 
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U.S. Army Institute for the 

Behavioral and Social Sciences 
5001 Elsenhower Avenue 
Alexandria, VA 22333 

Dr. Martin F. Wlskoff 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 

Dr. Dan Wolz 
AFHRL/MOE 

Brooks AFB, TX 78235 

Dr. Wallace Wulfeck, III 
Navy Personnel R&D Center 
San D1sgo, CA 92152-6800 
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